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ABSTRACT 

In the 2002-2003 academic year, a series of roundtable 
discussions was conducted to explore uses of data from the National Survey of 
Student Engagement (NSSE) for the improvement of student learning. This paper 
reports on one roundtable, held in June 2003 in Seattle, Washington. 
Participants included campus officers responsible for student assessment and 
institutional research and others with experience using NSSE data. The paper 
summarizes the key points about uses of the data, effective communication 
strategies, and ideas for further uses of NSSE data. NSSE data are being used 
to assess the first year experience, assess graduating seniors, and assess 
student experiences by academic department and major. The data have been 
embedded in larger improvement initiatives that include accreditation reports 
and reviews, agendas for improvement, and faculty participation in 
improvement. Campus leaders are referring to NSSE data in their work, and 
survey participants have developed good communication strategies to emphasize 
the portions of NSSE data that are most useful for a designated purpose. 

Among the suggestions for future uses of NSSE data are: (1) develop a 

comparison group of institutions; (2) develop a strategy for oversampling; 

(3) pay close attention to response rates; and (4) enlist student interest in 
and support of the NSSE survey. (SLD) 
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During the 2002-03 academic year, the American Association for Higher Education (AAHE) and 
the National Survey of Student Engagement (NSSE) conducted a series of roundtables to explore 
uses of NSSE data for the improvement of student learning. This roundtable series is part of the 
Documenting Effective Educational Practices project (Project DEEP), a partnership between 
AAHE and NSSE. 

A roundtable discussion held on June 22, 2003 at the AAHE Assessment Conference in Seattle, 
Washington included campus officers responsible for student assessment and for institutional re- 
search, and others with experience using NSSE data on their campuses. The discussion identified 
how NSSE data have been used, have been helpful, and have had an impact on campus. Partici- 
pants also discussed how NSSE data can best be shared with various members of the university 
community and ways to improve the uses of NSSE data. 

Almost all roundtable participants have had direct experience with NSSE data on their campuses. 
Because many had participated in NSSE surveys for two or more years, the discussion was rich 
with experience and ideas, blending stories on how NSSE data have been used on campus with 
su gg est i° ns for how the data can be used most effectively. Summarized below are key points 
about uses of the data, effective communication strategies, and ideas for further uses of NSSE 
data. 



NSSE data play a major role in campus efforts to assess student learning. 

Assessing the first-year experience. NSSE data are regularly used by many participants for assessing student 
experiences during their first year, a topic given priority attention on many campuses. The survey's focus on first- 
year respondents and the concreteness of the NSSE items are especially helpful for documenting students' actual 
experiences during the first year. Survey results have sometimes led to quick actions. At Plymouth State Univer- 
sity, NSSE data that highlighted problems with the first-semester Freshman Seminar were the basis for a redesign 
of the seminar in time for the next class of entering students. Sorting results for individual majors has also been 
fruitful: Brigham Young University found that students in certain pre-professional major programs were struggling 
with academic requirements more than other students during the first year. 

Assessing graduating seniors. NSSE data are valuable in understanding the undergraduate experience as re- 
ported by seniors. The University of Nebraska-Lincoln, which learned from NSSE data that its seniors have lower- 
than-acceptable scores on academic challenge, plans to explore this issue in greater depth. At Oklahoma State Uni- 
versity, NSSE data on graduating seniors were disaggregated to compare the expenences of seniors who had trans- 
ferred into the university with those who had been at the university for their entire experience. Close to half of the 
graduating seniors at OSU are transfer students. 

Assessing student experiences by academic department and major. Plymouth State University used NSSE 
data to conduct a special analysis of students “undecided” about their major. It subsequendy received funding from 
the Davis Foundation for a three-year project in which NSSE results will be used as part of campus efforts to help 
“undecided” students become more engaged. The College of Environmental Sciences at Oklahoma State Univer- 
sity holds a faculty retreat each fall that reviews the effectiveness of its programs. NSSE data, which are organized 
by department for this purpose, have become a major resource for discussion during the retreat. 
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NSSE data have been embedded in larger improvement initiatives. 

Accreditation reports and reviews increasingly make use of NSSE data. Both regional and specialized ac- 
creditation agencies expect campuses to monitor student learning and to document institutional efforts to improve 
quality. Several roundtable participants reported that NSSE data have become components of institutional self- 
analysis. For example, Norfolk State University plans to use several years of NSSE data in its analyses prepared for 
SACS accreditation reporting. 

NSSE data support many institution- wide agendas for improvement. The University of Wisconsin-Stevens 
Point has undertaken a broad initiative to support best practices in teaching. In an innovative teaching project de- 
signed to foster more stimulating teaching and active learning, NSSE data are regular components of project dis- 
cussions and experiments with different teaching approaches. Southern Illinois University-Edwardsville, a large 
campus with many commuting students, uses NSSE data in its formal orientation workshops for newly hired fac- 
ulty. As part of these workshops, NSSE data are highlighted to profile the university’s students and to describe the 
current teaching and learning environment on campus. 

The University of Nebraska -Lincoln used NSSE data to support the work of a Chancellor’s “Blue Sky” committee 
charged with setting new directions for die university. Using three sets of data — including the NSSE data, infor- 
mation from a survey of faculty and institutional climate, and data from the university’s work with performance/ 
quality indicators -- the Committee had many discussions and eventually generated a strong vision for promoting 
greater intellectual engagement among students. 

Plymouth State University is engaged in a long-term change project to develop best practices in engaging students 
for effective learning outcomes. NSSE data are tied to a detailed set of performance indicators being used on sev- 
eral topics. NSSE data also help in tracking progress with an institution-wide initiative to reform general education. 

NSSE data help encourage faculty participation in improvement efforts. Several institutions - the Univer- 
sity of Central Oklahoma, Plymouth State University, and the University of Nebraska-Lincoln - have used the Fac- 
ulty Survey of Student Engagement (FSSE). They found that the FSSE results are an effective complement to the 
NSSE results, particularly because they identify gaps between student reports and faculty perceptions of student 
experiences. FSSE results have helped convince faculty of the utility of NSSE results and have encouraged faculty 
members to reflect on their own teaching. 

Campus leaders are referring to NSSE data in their work. 

The Chancellor at Southern Illinois University-Edwardsville incorporated NSSE results in a recent state-of-the- 
university address. Findings from the data helped him to emphasize two key objectives for the university: to pro- 
duce “engaged students” and “capable grads.” 

At Norfolk State University, the Board of Trustees regularly receives reports based on the NSSE surveys. After a 
general presentation on the main results of the survey, board members requested more detailed data in later meet- 
ings. Members were pleased that the data offer them good information on the university and help the board to 
define areas for special attention. 

After reading the results carefully, the vice president for student affairs at The Evergreen State College focused on 
a few general themes needing closer attention. He then organized a number of meetings among deans and direc- 
tors to discuss the NSSE data and to explore weaknesses suggested by the pattern of results. In the productive 
dialogues that followed, discussants offered ideas about Evergreen’s students and ways that the campus environ- 
ment has an impact on student engagement. 

The Dean of the College of Environmental Science at Oklahoma State University frequently refers to NSSE data in 
remarks about the college and its effectiveness with teaching and learning. 



Participants have developed good communication strategies. 

Most roundtable participants have responsibilities for research and/or planning on their campuses; many serve as 
the coordinator of student assessment. Most help conduct the NSSE surveys, organize the development of any 
additional survey questions, and receive the full results. Strategies for disseminating results to other campus audi- 
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ences is thus a topic on which they have strong views and considerable experience. Several effective approaches 
emerged in the discussion. 

Customize the report to reach campus audiences. The person disseminating information should organize the 
NSSE results in a way that best addresses the most significant issues for the institution. Depending on campus 
culture, the focus of initial reports should be strengths; areas of weaknesses can be addressed in other particular 
settings. Oklahoma State University found that faculty accepted data more readily with the University’s own insig- 
nia and report formats. The Faculty Assessment Council helped interpret the data in well-understood on-campus 
vocabulary and contexts. The University of Charleston took another approach to “customizing” its NSSE results 
by developing tentative hypotheses about key areas of institutional priority in advance of receiving the results. 
Through this advance preparation, their review of the results had greater focus consistent with priorities. 

Organize NSSE data by individual colleges, departments, or programs. Different audiences will be ready to 
talk about and make changes at different times. For teaching and learning issues, discussions about NSSE data are 
best if they focus on smaller academic units, especially departments. The University of Wisconsin-Stevens Point 
was able to foster useful conversations about NSSE data through its teaching project, based in two of the univer- 
sity’s colleges. Similarly, administrators with specific program responsibilities — library services, residence living, 
academic advising or international students — are quick to see implications if data are disaggregated to highlight 
their areas of responsibility. Research librarians at the Evergreen State College requested a special discussion of 
NSSE data that was pertinent to the services they provide to students. 

Focus on the questionnaire items and percentages in the NSSE survey. The NSSE benchmarks are helpful 
organizing constructs for campus researchers, but they sometimes can be difficult to understand by general audi- 
ences unfamiliar with research on students. In contrast, campus audiences readily understand percentages and fre- 
quencies on specific questionnaire items. Plymouth State University selected those items in the NSSE data that 
stood out in helping to understand student experiences in general education courses, the focus of a major campus 
initiative. Southern Illinois University-Edwardsville found that the concreteness of the question items (e.g., the 
number of hours spent on each activity) offered an immediacy and relevance that effectively moved many faculty 
conversations toward generating ideas for change. 

Use the NSSE data as a starting point for exploratory conversations. As Brigham Young University reported, 
a realistic objective for initial meetings is to get conversations started and to get audiences to listen and reflect. 

Data, with detailed percentages and frequencies, can open discussion guided by general questions like “How do 
you know?” “Let’s see what the data show.” and “Do other items support the same interpretation?” Comparisons 
among several items should be encouraged, as certain interpretations make sense only when supported by patterns 
found in a good number of items. Several meetings or conversations may be needed before a group begins to 
make good use of NSSE results. The University of Central Oklahoma organized focus groups on three topics - 
student engagement, general education, and academic challenge — as a way to encourage discussion and found that 
these groups helped generate recommendations. 

Use the NSSE data to correct misunderstandings. With so many concrete items available, many ideas about 
what is right or wrong with current practices can be quickly tested. Washington State University found that some 
of their early conversations about NSSE results were useful in dispelling some myths about student experiences 
that had prevailed among various campus groups. WSU found that, contrary to expectations, its first-generation 
students were more satisfied than were other students. At Evergreen State College, NSSE data helped disabuse a 
myth that residence halls were detracting from the time students spent studying. 

Assemble and distribute hand-outs that offer detailed information on method and credibility of the survey. 

Questions about method inevitably come up, especially in first-time meetings. Users need such information, but it 
should not be allowed to side-track the attention to results. Detailed information is needed, but is more effectively 
conveyed through written explanations rather than by discussion, where each question tends to lead to another 
question. 

Suggestions about future uses of NSSE data 

Develop a comparison group of institutions. Comparing and discussing results for other campuses can help 
gready in interpreting the data for one’s own institution. Under NSSE procedures, institutions can select a specific 
set of “peer” institutions. For example. Southern Illinois University-Edwardsville, a member of the urban univer- 
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sity consortium, compared its NSSE results to other consortium members. Peer comparisons are especially helpful 
when the other institutions have some characteristics or concerns in common or when peers have been carefully 
selected to address issues of specific concern. 

Develop a strategy for oversampling. Roundtable participants consciously plan to over-sample different student 
groups in response to specific programmatic interests or priorities. Oversampling ensures that a sufficient number 
of students in those programs or groups have participated in the survey, making generalizations possible. Different 
groups can be oversampled in different years. Southern Illinois University-Edwardsville recently oversampled all 
first-year students enrolled in First Year Seminars in order to assess their experiences directly. SIU-Edwardsville has 
also over-sampled by college and by department, an approach that helps increase the interest of deans and depart- 
ment chairs in making use of the NSSE data. 

Pay close attention to response rates. Techniques for keeping response rates at acceptable levels should be care- 
fully maintained and improved, whenever necessary. As campuses incorporate uses of NSSE data into more of 
their regular monitoring and reporting on important aspects of institutional status, and as they increasingly rely on 
NSSE data for identifying trends, adequate response rates signal that the data offer reliable generalizations. NSSE 
makes it possible for schools to check their response rates on a weekly basis during the survey administration cycle. 

Enlist student interest in and support of the NSSE survey. Students can be very interested in exploring NSSE 
results. Their views are important and should be systematically sought, especially in discussing possible changes 
based on NSSE results. Student interest can also help encourage high levels of student participation in NSSE sur- 
veys. The University of Charleston met with student government to present NSSE data. Because student newspa- 
pers are a potential outlet, student reporters can be encouraged to develop stories on certain aspects of NSSE re- 
sults. At Norfolk State University, current students prepared stories about the NSSE results for the student paper. 
Student reports can be effective because they use students’ own words and perspectives to call attention to impor- 
tant findings. 

In sum, the roundtable discussions combined practical tips with creative approaches for building on participant 
experiences in using NSSE data. NSSE results have helped inform campus discussions and that student engage- 
ment results are part of continuing efforts to improve student experiences. 



The national roundtables are part of the Documenting Effective Educa- 
tional Practices (DEEP) Project ; a collaboration between the National 
Survey of Student Engagement and the American Association for Higher 
Education , with support from Lumina Foundation for Education . 
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